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Abstract— The Maximum Differential Backlog (MDB) 
control policy of Tassiulas and Ephremides has been shown 
to adaptively maximize the stable throughput of multi- 
hop wireless networks with random traffic arrivals and 
queueing. The practical implementation of the MDB policy 
in wireless networks with mutually interfering links, how- 
ever, requires the development of distributed optimization 
algorithms. Within the context of CDMA-based multi-hop 
wireless networks, we develop a set of node-based scaled 
gradient projection power control algorithms which solves 
the MDB optimization problem in a distributed manner 
using low communication overhead. As these algorithms 
require time to converge to a neighborhood of the optimum, 
the optimal rates determined by the MDB policy can only 
be found iteratively over time. For this, we show that 
the iterative MDB policy with convergence time remains 
throughput optimal. 

Index Terms — Throughput optimal control, multi-hop 
wireless networks, distributed optimization. 



I. Introduction 

The optimal control of multi-hop wireless networks is 
a major research and design challenge due, in part, to 
the interference between nodes, the time-varying nature 
of the communication channels, the energy limitation of 
mobile nodes, and the lack of centralized coordination. 
This problem is further complicated by the fact that 
data traffic in wireless networks often arrive at random 
instants into network buffers. Although a complete so- 
lution to the optimal control problem is still elusive, a 
major advance is made in the seminal work of Tassiulas 
and Ephremides [1]. In this work, the authors consider 
a stochastic multi-hop wireless network with random 
traffic arrivals and queueing, where the activation of links 
satisfies specified constraints reflecting, for instance, 
channel interference. For this network, the authors char- 
acterize the stability region, i.e. the set of all end-to-end 

1 This research is supported in part by Army Research Office (ARO) 
Young Investigator Program (YIP) grant DAAD19-03-1-0229 and by 
National Science Foundation (NSF) grant CCR-0313183. 



demands that the network can support. Moreover, they 
obtain a throughput optimal routing and link activation 
policy which stabilizes the network whenever the arrival 
rates are in the interior of the stability region, without 
a priori knowledge of arrival statistics. The throughput 
optimal policy operates on the Maximum Differential 
Backlog (MDB) principle, which essentially seeks to 
achieve load-balancing in the network. The MDB policy 
(sometimes called the "backpressure algorithm") has 
been extended to multi-hop networks with general ca- 
pacity constraints in [2] and has been combined with 
congestion control mechanisms in [3], [4]. 

While the MDB policy represents a remarkable 
achievement, there remains a significant difficulty in 
applying the policy to wireless networks. The mutual 
interference between wireless links imply that the evalu- 
ation of the MDB policy involves a centralized network 
optimization. This, however, is highly undesirable in 
wireless networks with limited transmission range and 
scarce battery resources. The call for distributed schedul- 
ing algorithms with guaranteed throughput gives rise to 
two main lines of research. 

One approach is to adopt simple physical and 
MAC layer models and apply computationally efficient 
scheduling rules in a distributed manner. The work in [5], 
[6] study networks where at any instant only mutually 
non-interfering links are activated, and any link, as long 
as it is active, transmits at a fixed rate. In particular, it 
is shown in [5] that Maximal Greedy Scheduling can 
achieve a guaranteed fraction of the maximum through- 
put region. This result is generalized in [6] to multi- 
hop networks where the end-to-end paths are given and 
fixed. Despite its simplicity, the distributed scheduling 
considered in the above work applies to only a limited 
class of networks. Moreover, the simplicity is gained at 
the expense of throughput optimality [7]. 

Another line of research develops distributed power 
control and rate allocation algorithms for implementing 



the MDB policy with the aim of preserving the through- 
put optimality. Thus far, distributed MDB control has 
been investigated only for networks with relatively sim- 
ple physical layer models. For example, Neely [8] studies 
a cell partitioned network model where different cells do 
not interference with each other so that scheduling can 
be decentralized to each cell. However, the question of 
how the MDB policy can be efficiently applied in general 
wireless networks remains unanswered. 

In this paper, we consider the implementation of 
the MDB algorithm within interference-limited CDMA 
wireless networks, where transmission on any given link 
potentially contends with interference from all other 
active links. In this setting, we present two main sets of 
results. First, we develop a set of node-based scaled gra- 
dient projection power control algorithms which solves 
the MDB optimization in a distributed manner using low 
communication overhead. As these algorithms require 
time to converge to a neighborhood of the optimum, 
the optimal rate allocation of the MDB policy can only 
be found iteratively over time. In the second result, we 
show that the iterative MDB policy with convergence 
time remains throughput optimal as long as the second 
moments of the traffic arrival rates are bounded. Com- 
bining these two results, we conclude that our algorithms 
yield a distributed solution to throughput optimal control 
of CDMA wireless networks with random traffic arrivals. 
The framework and techniques developed in this work 
can readily be adapted to general interference-limited 
stochastic wireless networks. 

Iterative implementation of the MDB policy has also 
been studied independently by Giannoulis et al. [9]. 
They investigate distributed power control algorithms 
for CDMA networks which are iterated once for every 
update of the queue state. In their scheme, the power and 
rate allocation algorithms are iterated only once in a slot, 
after which the queue state is re-sampled. In contrast, our 
algorithms re-sample the queue state only after the power 
and rate allocation has converged to the optimum for the 
previous queue state. We provide a rigorous proof for 
the throughput optimality of our scheme using a novel 
geometric method. We also compare the performance 
of our scheme with that of the non-convergent iterative 
algorithms in [9] and of the centralized MDB policy 
(whereby it is assumed that the optimal powers and 
rates are instantaneously obtained given the queue state) 
through simulations. In the experiments we conducted, 
the iterative MDB policy with convergence outperforms 
that without convergence. 

This paper is organized as follows. Section |ll] in- 
troduces the model of stochastic multi-hop wireless 



networks and reviews the throughput optimal Maximum 
Differential Backlog policy. The node-based power con- 
trol algorithms that achieve the optimal rate allocation 
are presented in Section [ni| In Section IIVI we prove 
that the iterative MDB policy proposed in the last section 
maintains the throughput optimality even in the presence 
of non-negligible convergence time. The comparison of 
different implementations of the MDB policy is con- 
ducted through simulations in Section IV1 

II. Network Model and 
Throughput Optimal Control 

A. Model of Stochastic Multi-hop Wireless Networks 

Consider a wireless network represented by a directed 
and connected graph Q = (Af,£). Each node i € J\f 
models a wireless transceiver. An edge € £ 

represents a unidirectional radio channel from node i 
to j. For convenience, let 0(i) = {j : E £} and 

T(i) = {j : (j, i) S £} denote the sets of node i's next- 
hop and previous-hop neighbors, respectively. Let the 
vector h — (/iy)(i,i)e£ represent the (constant) channel 
gains on all links. 

Denote the transmission power used on link at 
(continuous) time r by (r) > 0, and the instantaneous 
service rate of link by i?y(r) > 0. A feasible 

service rate vector R(t) — {R%j(T))u j\^e must belong 
to a given instantaneous feasible rate region C(P(t)) 
reflecting the physical-layer coding mechanism. Under 
peak power constraints Pj, i E Af, let 

n = J p( T ) eif: p y"( T ) ^ ^' Vi e N \ 

be the set of feasible power allocations and 

C(II) ^conv [ |J C(P) j 
\Pen / 

be the long-term feasible service rate region. Here, the 
convex hull operation conv(-) indicates the possibility of 
time sharing among different feasible power allocations 
Pell over a sufficiently long period. 

Let the data traffic in the network be classified ac- 
cording to their destinations. Traffic of type k E K. is 
destined for a set of nodes A4 C Af (when type k 
traffic reaches any node in Afk, it exits the network), 
where JC is the set of all traffic types. Let T > be 
a given time slot length. Let the number of bits of 
type k entering the network at node i from time tT 
to (t + 1)T be a nonnegative random variable B^[t]. 
Assume that for all t E Z+, B^[t] are independent 
and identically distributed with F(Bf [t] = 0) > 0. Let 
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EB k [t] = a$ < oo and E (B k [t]) 2 = b k < oo be the 
first and second moments of Bf[i\. Furthermore, assume 
all arrival processes i 6 Af,k € JC are 

mutually independent. 

Assume node i G Af provides a (separate) infinite 
buffer i k for each type k of traffic that is not destined for 
i. Denote the unfinished work in i k at time r by U k {r). 
We focus on the queue states sampled at slot boundaries 
t = tT, t G Z+. Let U k [t] denote the instantaneous 
backlog at the beginning of the tth slot, i.e., J7* [t] = 
Uf{tT). Over the tth slot, link serves i k at rate 

r(t+l)T pfe 



R k j[t] = J t y RfJr)dT. The aggregate service rate 
on link over the tth slot is i?y [i] = Sfcejc 
Thus, we have the following queueing dynamics: 



U?[t+1] < 

um- E E ^LM+^fw 

j'eO(i) m£Z(i) 

(1) 

Here denotes max{a;,0}, and the inequality comes 
from the fact that in general, since certain queues may 
be empty, the actual endogenous arrival rate is less than 
or equal to the nominal rate E roe x(i) R^M- 

B. Stability Region and Throughput Optimal Policy 

Given the wireless network model, we now define 
notions of stability and investigate throughput optimal 
control policies. 

Definition 1: [2] The queue i k is stable if gf(£) = 
Umsup^iELiP^fM >f] -> as £ ^ oo. 
Input processes {B[t] = (B k ! [t])ieAr,fceJc}tS:i are stabi- 
lizable if there exist service processes {i2y[£]} for all 
€ £ and k E JC such that for every t G Z + , 
G C(n), 2 and the resulting queueing processes are 
all stable. 

Definition 2: The stability region A of a wireless 
multi-hop network is the closure of the set of the average 
arrival rate vectors a of all stabilizable input processes. 

For a general wireless multi-hop network, its stability 
region has a simple characterization in terms of support- 
ing multi-commodity rates that are feasible under link 
capacity constraints. 

Theorem 1: [2] The stability region A of the wireless 
multi-hop network with transmission power constraint II 
is the set of all average rate vectors (a k ) such that there 

2 Here we assume the slot length T is long enough for time-sharing 
among different P S II. 



exists a multi-commodity flow vector (/,*) satisfying 
fii>0, V(i,j) G £ and k G K, 
E fiS~ E V.GAA, fcG/C, 



a k < 



jeO(i) 



iei(i) 



E fii ^ V(i, j) € £ where (C«)(ij)€£ e C(II). 
fee/C 

The following Maximum Differential Backlog (MDB) 
policy has been shown to be throughput optimal [1], 
[2] in the sense that it stabilizes all input processes 
with average rate vectors belonging to the interior of A, 
without knowledge of arrival statistics. The policy can 
be described as follows: 

1) At slot t, find traffic type k*j[t] having the 
maximum differential backlog over link 

for all G £. That is, k*j[i\ = 

argmax,. eJ c [U k [t] ~ U k [t}}, where U k [t] = If 
j G Af k . Let b*j[t] = max{0,Uf*[t}-Uf[t]}, 
where k* = 

2) Find the rate vector R*[t] which solves 



max E 6 «W ' Ri r 



Rec(n) 



(2) 



3) The service rate provided by link to queue 



is determined by 



4-M 



if fe 



otherwise. 



o, 

For wired networks, the above MDB policy can be 
implemented in a fully distributed manner. In wireless 
networks, however, the capacity of a link is usually 
affected by interference from other links. Therefore, 
solving (|2ji in general requires centralized computation. 
Thus far, distributed solutions for (|2ji are available only 
for relatively simple physical layer models [8]. 

In the following, we develop efficient distributed MDB 
control algorithms for interference-limited CDMA net- 
works with random traffic. Throughout the rest of the 
paper, we assume all nodes have synchronized clocks so 
that the boundaries of time slots at all nodes are aligned. 
This assumption guarantees that the MDB values in Q 
are taken at the same instant across all links. The study 
of MDB policy based on asynchronously sampled queue 
state will be a subject of future work. 

III. Distributed Maximum Differential 
Backlog Control 

A. Throughput Optimal Power Control 

We study a wireless network using direct-sequence 
spread-spectrum CDMA. The received signal-to- 
interference-plus-noise ratio (SINR) per channel code 
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symbol of link is given by 

SINE = KhjjPjj 

v 6^ {Pi - Pij) + £ h mj P m + Nj ' 

where K is the processing gain, P m = J2keO(m) p mk 
is the total transmission power of node m, and Nj 
represents the noise power of receiver j. The parameter 
Oi £ [0, 1] characterizes the degree of self-interference. 3 
Assume the receiver of every link decodes its own sig- 
nal against the interference from other links as Gaussian 
noise. The information-theoretic capacity of link is 
given by 



R s log 



/ 



V 



KhijPij 



\ 



m^i 



For convenience, we normalize the channel symbol rate 
R s to be one for subsequent analysis. We also take log(-) 
to be the natural logarithm to simplify differentiation 
operations. 

In most CDMA systems, due to the large multiplica- 
tion factor K, the SINR per symbol 

Khij Pij 



^ihij(Pi Rij) ~f~ hmj-Prr 
m^i 



Nj 



is typically high [10]. Therefore, in the high SINR 
regime, we can approximate the capacity of any active 
link by 

/ 



lo. 



g 



KhijPij 



. Oihij J2k^j + Zj hmj J2keO(m) R"ik + Nj 



With a change of variables Si — In Pi, Si = InP;, 
and Sik = InPifc, the capacity function becomes 

C i j(S)=log(Kh i j)+S ij - 



log ( Bihij eS " 



Ni 



which is known to be concave in S [11], [12]. 
It follows that the instantaneous achievable region 
lJ J 3 gn C(P) is convex, and therefore is equal to C(H) = 
conv(Up en C(P)). 

3 8i = corresponds to the case when node i applies mutually 
orthogonal direct sequences for transmissions to its receivers. In this 
case, signals intended for different receivers will not interfere with each 
other in demodulation. The other extreme, where 0j = 1, represents 
the most pessimistic case where self-interference is as significant as 
all other sources of interference. 



Thus, the optimization problem in (0 at a fixed 
time slot can be seen as optimizing over the region 4 
{J Pen C(P). More specifically, it can be rewritten as 
the following concave maximization problem 



maximize KjRij (3) 

subject to Rij = Cij(S), V(i, j) G £, 
Y e StJ <Pi, VieJV. 

Without loss of generality, we assume &*• > for 
all (otherwise we can simply exclude those links 

having b*j = from the objective function in l|3}). 

B. Power Adjustment Variables 

Next we introduce a set of node-based control vari- 
ables for adjusting the transmission powers on all links. 
They are 

Power allocation variables: rjik = (h k) € £, 

Ri 

S- 

Power control variables: 7i = — , i£ A/". 

Si 

These variables are illustrated in Figure □ With appro- 




R,-R>L 



Fig. 1 . Transmission powers in terms of the power control and power 
allocation variables. 



priate scaling, we can always let P; > 1 for alH € J\f so 
that Si > 0. Therefore, we have the following equivalent 
' Throughput Optimal Power Control (TOPC) problem: 



maximize 



"ij 

(id) 



bt log ■ 



KhijiPi^rjij 



hij(R)^(l-V^) + 



h (P I 7 " 



Nj 



(4) 



4 Notice that even if (J p^nC(P) is not convex, restricting the 
feasible set of the optimization in |2) to Up,g n C(.P ) does not lose 
any optimality. This is because the objective function is linear in the 
link rates, and so the maximum over any compact region region is 
equal to the maximum over the convex hull of that region. 
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subject to rjij > 0, V(i,j) G £, 

Vij = 1, 7i < 1, Vi G TV. 

C. Conditions for Optimality 

To solve the TOPC problem in @, we compute the 
gradients of the objective function, denoted by F, with 
respect to the power allocation variables and the power 
control variables, respectively. They are as follows. For 

all i G TV and j G 0(i), 



dF 



= Pi 



E 

fceo(i) 



IN,, 



E E 6 " lfc 



IAT m , 



where the power allocation marginal gain indicator is 



" 6 « (77- + 



^2 ^"ZJ 



(5) 



For all i G TV, 



<9F 



5*4 • Sji, 



where the power control marginal gain indicator is 



H = Pi 



E E 

m=fii kGO(rr, 



- b rnk h *k 
IN m k 



+ 



E 

keO(i) 



IN ik 



E ^tk ■ Vik 



■(6) 



The term INy appearing above is short-hand notation 
for the overall interference-plus-noise power at the re- 
ceiver end of link (i,j), that is 



IN i:j = 6ih i:j Y e Slfc + E h ™3 E eS " 



keO(rn) 



The marginal gain indicators fully characterize the 
optimality conditions as follows. 

Theorem 2: A feasible set of transmission power vari- 
ables {i]ik}(i.k)e£ an d {7i}zG7V is the solution of the 
TOPC problem if and only if the following conditions 
hold. For all i G Af, there exists a constant z/j such that 

<% fe = Vfc G 0(i), (7) 

8 lt = 0, if 7i < 1, (8) 
H > 0, if 7i = 1. (9) 

Here, all > since > by assumption. 



For the detailed proof of Theorem |2] see [13]. Due 
to the distributed form of the optimality conditions, 
every node can check the conditions with respect to 
its controlled variables locally, and adjust them towards 
the optimum. In the next section, we present a set of 
distributed algorithms that achieve the globally optimal 
power configuration. 

D. Distributed Power Control Algorithms 

We design scaled gradient projection algorithms which 
iteratively update the nodes' power allocation variables 
and power control variables in a distributed manner, so as 
to asymptotically converge to the optimal solution of 
At each iteration, the variables are updated in the positive 
gradient direction, scaled by a positive definite matrix. 
When an update leads to a point outside the feasible set, 
the point is projected back into the feasible set [14]. 

1) Power Allocation Algorithm (PA): At the fcth iter- 
ation at node i, the current local power allocation vector 
»7i = (Vij)jeo(i) is updated by 

V k+1 = PA(r,*) = [rii+Pi ■ (Qi)' 1 • S V k ] + . 

Here, 5r] k = (5??y)ieO(i) an d Pi is a positive stepsize. 
The matrix Q k is symmetric, positive definite on the 



subspace {vi : 



ieO(t) Vi 3 



0}. Finally, [']i. k denotes 



the projection on the feasible set of rj i relative to the 
norm induced by Q k . 5 

Suppose each node j can measure the value of 
SINRij for any of its incoming links. Before an it- 
eration of PA, node i collects the current SINRij 's via 
feedback from its next-hop neighbors j. Node i can then 
readily compute all Srjij's according to 



{pa ' 




= b A( 




INij) 


Pij \ 



1 



1 SINR, 
K 



Note that since the calculation of Srjij involves only 
locally obtainable measures, the PA algorithm does not 
require global exchange of control messages. 

2) Power Control Algorithm (PC): After a phase for 
exchanging control messages (which will be discussed 
below), every node i is able to calculate its power 
control marginal gain indicator Sji. From a network- 
wide viewpoint, the power control vector j k = {j k )i^ 
is updated by 

j k+i = pc( 7 fc ) = b k +e ■ (v'r 1 ■ sj k ]y k . 

5 In general, [i]"t fc = arg min X £jr(x — x)' ■ Q k ■ (x — x), where 

^ i 

T is the feasible set of x. 
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Here, £ is a positive stepsize and matrix V is sym- 
metric and positive definite. Note that PC becomes 
amenable to distributed implementation if and only if 
V k is diagonal. 

We now derive an efficient protocol which allows 
each node to calculate its own 6ji given limited control 
messaging. We first re-order the summations on the RHS 
of l|6} as 



H. = Pi 



E 



E 

meX(j) 



IN mi 



P 



+ (9^ - Oi + 1) 



With reference to the above expression, we propose the 
following procedure for computing 6ji. 

Power Control Message Exchange Protocol: Let each 
node j assemble the measures from all its incom- 

ing links (m,j). For this purpose, an upstream neighbor 
to needs to inform j of the value b* m jj P m j. Since node 
j can measure both SINR m j and h m j, it can calculate 



b* ■ 



Knj SINRmj 



JN P 



h K 

1 ir mj 1 Y 



After obtaining the measures from all incoming links, 
node j sums them up to form the power control message: 



Msg(j)i= J2 



mexu) INm ° 

It then broadcasts Msg(j) to the whole network. The 
process for control messaging is illustrated by Figure [2] 
where the solid arrows represent local message commu- 
nication and the hollow arrow signifies the broadcasting 
of the message. 



Msg{j)= E 




me/a) IN mj 



Fig. 2. Information Exchange Protocol for Power Control Algorithm 



Upon obtaining Msg(j) from node j ^ i, node i 
processes it according to the following rule. If j is a 
next-hop neighbor of i, it multiplies the message with 
hij and subtracts the product from the local measure 



1 

P 



■ (OiVij - 
Srjij ■ rjij + Sriij 



IN,,. 



b*- 
P 

J - I '1 



Otherwise, it multiplies M sg(j) with — hij. Finally, node 
i adds up the results derived from processing all other 
nodes' messages, and this sum multiplied by Pj equals 
Sji. Note that in a symmetric duplex channel, hij ~ hji, 
and node i may use its own measure of hji in place of 
hij . Otherwise, it will need channel feedback from node 
j to calculate hij. To summarize, the protocol requires 
only one message from each node to be broadcast to the 
whole network. 



3) Convergence of Algorithms: We now formally 
state the central convergence result for the PA and PC 
algorithms discussed above. 

Theorem 3: From any feasible initial transmission 
power configuration and 7 , there exist valid 

scaling matrices {Q^} and V k , and positive stepsizes 
{/Sf} and £ fc such that the sequences generated by the 
algorithms PA(-) and PC(-) converge, i.e., rj k — > 77* 
for all i, and 7 fe ^7*asfc^oo. Furthermore, {rj* } 
and 7* constitute a set of jointly optimal solution to the 
TOPC problem @. 

In the PA and PC algorithms, the scaling matrices 
are chosen to be appropriate diagonal matrices which 
approximate the relevant Hessians such that the objective 
value is increased by every iteration until the optimum 
is achieved. This allows the scaled gradient projection 
algorithms to approximate constrained Newton algo- 
rithms, which are known to have fast convergence rates. 
Furthermore, the scaling matrices are shown to be easily 
calculated at each node using very limited control mes- 
saging. The detailed derivation of these parameters and 
the full proof of Theorem [5] can be found in [13]. 

Also note that the convergence of the algorithms does 
not require any particular order of running PA and PC 
algorithms at different nodes. Any node i only needs to 
update its own variables 77^ and ji using PA and PC 
until its local variables satisfy the optimality conditions 
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IV. Throughput Optimality of Iterative 
Maximum Differential Backlog Policy 

Since the PA and PC algorithms need a certain num- 
ber of iterations before reaching a close neighborhood of 
an optimum to the problem in @, the optimal service 
rates dictated by the MDB policy cannot be applied 
instantaneously. Rather the optimal service rates can only 
be found iteratively over time. At any moment in the 
convergence interval, the queues are served at the rates 
which are iteratively updated towards the optimal rates 
for the queue state at the beginning of a slot. The service 
rates obtained at the end of a convergence period are 
optimal only for the queue state some time ago. The 
effect of using lagging optimal service rates is studied 
in the context of N x N packet switches by Neely 
et al. [15] 6 and in a queueing network with Poisson 
arrivals and exponential service rates by Tassiulas and 
Ephremides [16]. In [15], [16], however, the process of 
finding the optimal rates is not iterative. It is assumed 
that once the (outdated) queue state information becomes 
available, the optimal rates are obtained instantaneously. 
Here, we analyze the iterative MDB algorithm with 
convergence time in general multi-hop networks with 
i.i.d. random arrival processes and general rate regions. 
We show that the throughput optimality of the MDB 
policy is preserved for any finite convergence time. For 
this, we invent a new geometric approach for computing 
the expected Lyapunov drift of the queue state. 

A. Transient Optimal Rates 

Without loss of generality, assume the convergence 
time of the MDB algorithms in Section IIII-DI is the 
length of a time slot T, 1 i.e., at time r = (t + 1)T, 
the optimal service rate vector for U[t] is achieved. For 
ease of analysis, we further scale time so that T = 1. 

We assume a general feasible service rate region. 
Instead of studying the service rates (i?^(r)), in this 
section we focus on the virtual service rates. First define 
the instantaneous virtual service rate of queue i k by 8 

jeo{i) mex(i) 

6 In [15], the current queue state is taken to be the state of the 
Markov chain used for stability analysis. As we show below, however, 
the Markov state should consist of the current queue state as well as 
the previous queue state. 

7 In practice, the gradient projection algorithms can only find an 
approximate optimal solution within a finite period of time. In this 
work, we make the idealization that the exact optimum can be achieved 
after the convergence period T. Such an assumption simplifies the 
following analysis while its loss of precision is small when we take T 
sufficiently large. 

8 Virtual service rates can be negative, as when a queue's endogenous 
incoming rate is higher than its outgoing rate. 



Such a transformation considerably simplifies our sub- 
sequent analysis. The vector of virtual service rates over 
the tth slot is 

rt+i 

R[t] = J R(r)dr, 

where the integration is taken component-wise. By 
definition, we have Rf[t] = J2jeo(i) -^j 'W ~ 
EmeiM^LM- Therefore, we consider R[t] = 
(R k [t])ieAf,k£ic induced by R[t]. A virtual service rate 
vector over a slot R[t] is feasible if it is induced by a 
feasible R[t] € C(il). Denote the set of all feasible R[t] 
by C^(LI). It is straightforward to verify that C^(II) is 
compact and convex. By Theorem 1 of [3], the subset of 
Cr(II) in the positive orthant is the stability region of 
the wireless multi-hop networks with power constraints 
II. For brevity, we denote Cr(LT) by C in this section. 
Finally, the queueing dynamics in Q can be written in 
vector form as 

U[t+l]<(u[t]-R[t] + B[t]\ + . (10) 

Note that maximizing the MDB objective function 
(0 in R over the feasible service rate region C(H) is 
equivalent to maximizing U[t]' ■ R in R over the virtual 
service rate region C. We denote the maximizing R by 
R (U[t\). From now on, we simply call R the service 
rate vector and refer to R (U[i\) as the optimal rate 
allocation for queue state U[i\. 

Recall our discussion of the distributed MDB control 
algorithms in the last section. Due to the iterative nature 
of the algorithms, the optimal power vector and the 
optimal rate allocation for a given queue state can be 
found only when the algorithms converge. Therefore in 
practice, the rate vector solving (|2j for (b*j[t]) cannot 
be applied instantly at the beginning of the tth slot. 
The actual service rates R(t),t 6 R + , are always in 
transience, shifting from the previous optimum to the 
next optimum. Thus, the instantaneous rate vector at time 
t = t is R(t) = R*(U[t - 1]), and at time r = t + 1, 
R(t+1) = R*(U[t}). 

B. Lyapunov Drift Criterion 

Following the previous model, the process 
{{U[i\,U[t - l])}^ =1 forms a Markov chain. The 
state (U[t], U[t — 1]) = W[t] lies in the state space 
W = K+ x M| 7 where M is the total number of 
queues. As an extension of Foster's criterion for a 
recurrent Markov chain [17], the following condition 
is used in studying the stability of stochastic queueing 
systems [1], [15]. 



7 



Lemma 1: [1] If there exist a (Lyapunov) function 
V : W h-> R+, a compact subset Wo C W, and a 
positive constant £o such that for all w 6 Wo 

E[y(w[t + i])-y(w[t])|w[t] =t»] <oo, (ii) 

and for all ^ Wo 

E [V(W[t + 1]) - V(W[t])|W[t] =w}< -e 0) (12) 

then the Markov chain {W[t]} is recurrent. 9 Hence, the 
queueing system is stable in the sense of Definition ^ 

We use the Lyapunov function from [16]: 

v(w[t]) = ^JW + ^M-^-i])' 

= ||E7[t]|| a + ||l/[t]-I/[t-l]|| a , 

where || ■ || denotes the L 2 norm. Using relation il OK we 
derive the following upper bound on the expected one- 
step Lyapunov drift conditioned on W[t] = (u t ,u t -i): 

E[V(W[t + l])-V(W[t])\W[t] = (u u u t ^)] 

< 2u' t (a-R[tty + 2(\b\ + \\R[t] || 2 ) 

-\\u t - M t -l|| 2 , 

where b is the vector of second moments of the random 
arrival rates and | • | denotes the L 1 norm. The detailed 
derivation of the above inequality is left to Appendix 1X1 
Because the distributed power adjustment algorithms 
in Section UlI-DI increase the objective value u' t ■ R with 
every iteration from time t to u' t -R(r) is increasing 
in t e [t,t+ 1) and given W[t] = (u t , «t-i), 

rt+l 

u' t ■ R[t] = u' t - R(r)dT > 

rt+l 

J u' t - R(t)dr = u' t - R(t) = u' t - R*(u t -i). 

Also notice that because the second moment vector b 
is assumed to be finite and R[t] lies in the bounded 
region C, we can find a finite constant A such that 
2 (\b\ + \\R[t]\\ 2 ^j < A. Thus, the conditional expected 
Lyapunov drift is upper bounded by 

2u' t ■ (a - R*(u t -i)) - \\u t - u t -i\\ 2 + A. 

Using the above Lyapunov function and the upper 
bound for the expected Lyapunov drift, we show the 
following main result. 

9 For a Markov chain with continuous state space to be recurrent, 
the following condition usually is required in addition to those in 
LemmaQ there exists a subset of states which can be visited from any 
other state (in a finite number of steps) with positive probability. For 
{W[t]} studied here, the zero state constitutes such a subset because 
by assumption V(Bf[t] = 0) > for all queues i k . 



Theorem 4: The iterative MDB policy with conver- 
gence time is throughput optimal, i.e. it stabilizes all 
arrival processes whose average rate vector a € int C. 

Guided by the Lyapunov drift criterion, the proof 
aims to find an Eq > and a compact set Wo (which 
may depend on Eq) which satisfy the conditions il 1> - 
O for any average arrival rates a £ int C. Note 
that condition il It is always satisfied since the first and 
second moments of arrival rates as well as the service 
rate vector are bounded. Now consider the compact 
region characterized by 

Wo = {w e R+ x Ef : V{w) < Q}. (13) 

Given £o > 0, we need to specify a finite £1 and show 
that when w[t] = (u t ,u t -i) ^ Wo, 

2«;-(a-fi*(w t _ 1 ))-||u t --it t _ 1 || 2 + A < -e . (14) 

Towards this objective, we devise a geometric method 
to relate the position of u t and Ut-i in the state space 
to the value of the inner product u' t -[a — R (it t _i)]. In 
order to reveal the insight underlying this approach, we 
first develop the methodology in K 2 . The generalization 
to higher dimensions as well as the proof for Theorem 
can be found in Appendix |B] and [C] 

C. Geometric Analysis 

In this section, we analyze vectors of arrival rates, 
service rates, and queue states geometrically. In view of 
condition ill 4b . we characterize a neighborhood around 
u t which has the following properties: if u t -\ lies in the 
neighborhood, then the first term 2u' t ■ (a — R (ut-i)) 
is substantially negative (< —A — Eq); if u t -i lies 
outside the neighborhood (meaning that \\u t — itt_i|| 2 is 
relatively large), then the second term —\\u t — Mt_i|| 2 
is sufficiently negative for dl4> to hold. 

We assume an average arrival rate vector a £ int C. 
There must exist a point a £ bd C, and a positive 
constant e such that a + e • 1 < a. Therefore the point 
e = a + | • 1 is also in the interior of C. 

Given the current queue state vector u t > 0, the 
hyperplane B e (u t ) = {x : u' t ■ x = u' t ■ e} is perpendic- 
ular to u t and crosses the point e. The intersection of 
halfspace (ut) — {x : u' t -x > u' t -e] withC, denoted 
by C+(it t ), is closed and convex with non-empty interior 
[18]. 

Lemma 2: For y £ C^(ut), u' t ■ [a — y] < —^\\u t \\. 

Proof: Since y £ Ti.^(u t ), by definition u' t -y > u' t -e. 
Thus, 

u't ■ [a — y] < u't ■ [a — e] 
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= -^\u t \<--\\u t \\. 

The last inequality follows from \u t \ > \\u t \\ since 
u t > 0. □ 

Two-Dimensional Heuristic: Assume there are two 
queues in the network and index them by 1 and 2. In this 
subsection, all vectors, hyperplanes, surfaces, etc. are in 
M 2 . The hyperplane B e (u t ) must intersect bd C at two 
different points, as illustrated in Figure [3] Let the two 




Fig. 3. The geometry when B e (ut) intersects bd C at two different 
points in RjJ" 

points be f 1 and / 2 , where f 1 is the upper-left one. 
Denote the hyperplane (which is a line in M 2 ) tangent 10 
to C at f l by Bf^rix), where n x is the unit normal 
vector of the tangent line. Specifically, we require n x 
to be pointing outward from C. Since C is not confined 
in f l is not necessarily nonnegative, and neither is 
rii. If there exist multiple tangent lines at f x , take ri\ 
to be any one of them. Let the unit normal vector at f 2 
be rt2, defined in the same manner. Let 

6*i (ul) — arccos(n' 1 • u~l), #2(^4) = arccos(n 2 ■ 

where n't stands for the normalized vector of u t . Since 
e 6 int C, ri\ and n-2 can never be parallel to u[. Thus, 

ti^ • Ut < 1, TI2 • Ut < 1, 

and 6\ (ul) > 0, 02(ut) > 0. Moreover, #i(it^) and 
02(ut) are bounded away from zero for all u t . To see 
this, we make use of Figure [3] again. The point f e is 
on the boundary and the vector f e — e is parallel to 
u t . By simple geometry, the convexity of the rate region 
implies 9i(u t ) > arctan(||/ e — e|| /|| — e||). Because 
e is an interior point, ||/ e — e|| > £ > 0. Moreover, 

'°The tangent hyperplane contains f 1 and defines a halfspace 
containing C. 



II /1 — e l < D < 00 since C is a bounded region. 
Therefore, 9i(ul) > arctan(^/Z?) > 0. The same is true 
for 6*2(1*4). Thus, we can construct a non-empty cone 
emanating from the origin sweeping from the direction 
of vector u t clockwise by 6>2(Mt) and counterclockwise 
by 6i(u t ). Such a cone always contains u t in its strict 
interior. This is illustrated in Figure |4] 

We consider the following two cases. First, if 
||ti t -ix t _i||/||w t || < sin [min{6ii(i^),6l2(i^),7r/2}] = 
a(u t ), then the pair of points (u t ,u t -i) both lie in the 
cone described above. In this case, u t -i is said to be in 
the neighborhood of u t . See Figure 




Fig. 4. The geometry of ut-i lying in the neighborhood of ut, 
where r = ||tit|| ■ a(ut). 

Let a be the infimum of a(ut) over all nonnegative 
unit vector ~u t . Because all 9i(u t ) and 62 (ui) are 
strictly positive, a must be strictly positive. If ||it t — 
iit_i||/||u t || < a, u t -i is also in the cone with u t . In 
this case, the hyperplane of normal vector u t -\ tangent 
to the rate region C touches bd C at R («t-i) some- 
where between / x and / 2 , i.e., R (ttt-i) £ C^(u t ). 
By Lemma 13 the inner product u' t ■ [a — R (u t -i)\ < 
-f || Mt||. Then for all w[t] such that V(w[t}) > (1+a 2 )- 
(e + A) 2 /e 2 = fii, ||tt t || > (e + A)/e, and therefore 

2u' t ■ (a- R {ut-ifj - \\u t - itt-i|| 2 + A 

< 2u' t ■ (a - R* {ut-^ +A < -e , 

which is the desired condition dl4> . 

If ||ti t -w t _i||/||w t || > a and assume ||w t -M t _i|| 2 = 
u>, then 

2u' t ■ (a - R*(u t -if) - \\u t - Mt-i|| 2 + A 

< 2||w t ||||a - R*(u t -i)\\ - w + A 

< 2^uj/u 2 yf\/2 - w + A 
= v2oj\/a — to + A. 
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Define 



L02 



inf{w > : \2uj\/a — u> + X < -£q}. (15) 



Then for all such that V(w[t]) > (1 + 1/V)w 2 = 
1^2, 1 1 tit - tt t _i|| 2 > u> 2 and {14} holds. 

Combining the above two cases and letting 51 = 
max{fli, O2}. we see that the region specified in (1131 
satisfies Lemma Q an d Theorem |4] follows. 

V. Numerical Experiments 

To assess the practical performance of the node-based 
distributed MDB policy in stochastic wireless networks, 
we conduct the following simulation to compare the total 
backlogs resulting from the same arrival processes under 
different MDB schemes. 

Our scheme iteratively adjusts the transmission powers 
during a slot to find the optimal rates for the queue state 
at the beginning of a slot. As a consequence, the MDB 
optimization is done with delayed queue state informa- 
tion, the transmission rates keep changing with time, and 
the optimal rates are achieved only at the end (beginning) 
of the current (next) slot. Recently, Giannoulis et al. [9] 
proposed another distributed power control algorithm to 
implement the MDB policy in CDMA networks. Instead 
of converging to the optimal solution for the current 
MDB problem, their scheme updates the link powers 
based on the present queue state only once in a slot. 
The new queue state at the beginning of the next slot 
is used for the subsequent iteration. To highlight the 
above difference, we refer to our method as "iterative 
MDB with convergence", and the method studied in [9] 
as "iterative MDB without convergence". Both schemes 
are shown to preserve the throughput optimality of the 
original MDB policy, which ideally (instantaneously) 
finds the optimal transmission rates for the queue state at 
the beginning of a slot, and applies them for the whole 
slot. 

For a single run of the experiment, we use a network 
with N nodes uniformly distributed in a disc of unit 
radius. Nodes i and j share a link if their distance d(i,j) 
is less than 2.5/viV, so that the average number of a 
node's neighbors remains constant with N. The path gain 
is modeled as hij — d{i 1 j)~ A . The processing gain of 
the CDMA system is K — 10 5 , and the self-interference 
parameter is 6i — 0.25. All nodes are subject to the 
common total power constraint Pi = 100 and AWGN of 
power Ni = 0.1. 

Each node is the source node of one session with the 
destination chosen from the other A — 1 nodes at random. 
At the beginning of every slot, the new arrivals of all N 
sessions are independent Poisson random variables with 



the same parameter B. As an approximation, we assume 
the iterative MDB scheme converges after 50 iterations 
of the PA and PC algorithms. The convergence time is 
taken to be the length of a slot, as in Section ITVl The 
network performance is investigated under each of the 
MDB schemes with the same set of arrival processes. 
The total backlog in the network is recorded after every 
slot. Figure |3 shows the backlog curves generated by the 
three schemes after averaging 10 independent runs with 
the parameters N = 10 and B = 4. Figure [6] reports the 

Total Backlog Under MDB Schemes (N = 10, B = 4, 10 Trials) 
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Fig. 5. Total backlogs under three MDB schemes (N = 10, B = 4). 

result from the experiment with the parameters N = 5 
and B = 7. The three methods all manage to stabilize the 

Total Backog Under MDB Schemes (N = 5, B = 7, 10 Trials) 



400 
f 350 

.Q 

•6 300 
H 

250 
200 
150 



o fO 



** *** x >ft " 
***fco,** * *OO x * 



6°? x o®2 



9°*** SxO* 



Ox*Q9 
xOfJ 



x iterative MDB with convergence - 
* iterative MDB w/o convergence 
o instantaneous MDB 



2.5 
x10 5 



Fig. 6. Total backlogs under three MDB schemes (TV = 5, B = 7). 

network queues in the long run. However, the iterative 
MDB scheme with convergence and the instantaneous 
MDB scheme result in lower queue occupancy, hence 
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lower delay, than the iterative MDB scheme without 
convergence. 

VI. Conclusion 

In this work, we study the distributed implementa- 
tion of the Maximum Differential Backlog algorithm 
within interference-limited CDMA wireless networks 
with random traffic arrivals. In the first half of the 
paper, we develop a set of node-based iterative power 
allocation and power control algorithms for solving the 
MDB optimization problem. Our algorithms are based 
on the scaled gradient projection method. We show that 
the algorithms can solve the MDB optimization in a 
distributed manner using low communication overhead. 
Because these iterative algorithms typically require non- 
negligible time to converge, the optimal rate allocation 
can only be found iteratively over time. In the second 
half of the paper, we analyze the iterative MDB policy 
with convergence time. Using a new geometric approach 
for analysis of the expected Lyapunov drift, we prove 
that throughput optimality of the MDB algorithm still 
holds as long as the second moments of traffic arrival 
rates are bounded. The two parts of the paper in conjunc- 
tion yield a distributed solution to throughput optimal 
control of CDMA wireless networks with random traffic 
arrivals. 

Appendix 

A. Derivation of Lyapunov Drift 

By definition, the difference of Lyapunov values 
V(W[t + 1]) and V{W[t\) can be written as 

V(W[t + l])-V{W[t]) 

= \\u[t + i]f -\\u[t]f + \\u[t + i]-u[t]f 

-\\U[t]-U[t-l]f 

= 2u[t + iy • (u[t + 1] - u[t\) 

-\\U[t]-U[t-l]\\*. 
Using relation i ll Oi l, we have 

U[t + l]'-(U[t + l]-U[t]) 

< (fu[t]-R[t]+B[t]) + '\ ■ 
(fu[t]-R[t]+B[t]) + -U[t]\ 

< (u[t]-R[t] + B[t})' -(B[t}-R[t}) 

< U[t]'.(B[t]-R[t]) + \\B[t}\\ 2 + \\R[t]f. 
Therefore, we finally obtain 

V(W[t + l])-V(W[t]) 



< 2U[t]'-(B[t}-R[t}) + 2(\\B[t}f + \\R[t}f) 

-\\u[t]-u[t-iW. 

B. Geometric Analysis in M. 

We now generalize our geometric analysis in Sec- 
tion IIV-CI to M-dimensional space. We retain the no- 
tation from Section llV-CI 

Analogous to the argument used in the two- 
dimensional case, we focus on characterizing the neigh- 
borhood of u t . 

Lemma 3: For any u t > 0, there exists a region 
JC(u t )C R+ such that 

1. u t G JC(u t ); 

2. K(ut) has non-empty and convex interior relative 
to any one-dimensional affine space containing it t ; 

3. For all Ut-i G JC(u t ), the optimal rate vector 
R (ut-x) with respect to Ut-i is in C+(w f ). 

Note that K.(u t ) is the M-dimensional analogue of 
the circle S(u t ,r) of radius r around u t in Figure 0] 
To facilitate the proof, define the set of feasible unit 
incremental vectors around a nonnegative unit vector u 
as 

A- 4 {A=(A 1 ,---,A M ) : 

|| A|| = 1, and Af > if = 0} . 

Proof of Lemma ]3\ Each A G A^ spans a one- 
dimensional affine space containing u t . It is sufficient 
to show that given any A G A^, there exists 8 > 
such that for all 8 G [0, 8} and / G C satisfying 

(u t + 8 A)' ■ f > (ut + SA)' ■ R, Vi?GC, (16) 

we have / £ C+(u f ). 

We prove the claim by construction. We make use of 
the dominant point a of a such that a + e ■ 1 < a (also 
e + e/2 ■ 1 < a). Define the parameter 

d(A) =maxA'- (R-a), (17) 

which is at least zero (by setting R = a in the objective 
function). It is possibly equal to zero, and must be 
bounded from above, because A is a unit vector and 
the optimization region C is compact. 
Now consider 

■= e\\u t \\ 
2(1^)' 

which by the above analysis is positive. Because C is 
convex and compact, for any 8 G [0,8] there exists at 
least one / satisfying d!6t . Picking any one such / and 
specifically letting R = a on the RHS of dl6l l. we have 

(u t + 8 A)' ■ f > (u t + 8 A)' ■ a. 
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By using the inequality 



£ £ 

u' t ■ a > u' t ■ e + -\u t \ >« t -e+-||«t||, 



we have 



> u\ ■e + ~\\u t \\-6A'.(f-a) 



> u' t ■ e 



Ut\\ — 6 max A' • (R — a) 



Rec 

g||Mt|| 

2d(A) 



•d(A) 



Thus, we can conclude that / S C£(ut). Since / is 
chosen arbitrarily, the claim at the beginning of the proof 
is proved. 

Finally, define JC(ut) as 



Ut-i elf : \\u t -i-ut\\ < 



s\\u t \ 



2d(u t -i - u t ) 



(18) 

where d(-) is defined as in d!7l >. To accommodate 
the special case of Ut-i = tit, we define d(0) = 0. 
It is easily verified that the so-constructed fC(ut) is 
a valid neighborhood of Ut, as required by the lemma. □ 



C. Proof for Theorem^ 
If 



\Ut-i - u t \ 
IKII 



< 



2sup^ >0 d(M) 



then Ut-i € IC(u t ) where JC(u t ) is defined in Jl 81 . In 
this case, for all w[t] such that V («>[<;]) > (1 + a 2 ) ■ 
(eo + A) 2 /e 2 = Qi, \\u t \\ > (e + A)/e, and therefore 



2uJ • (a - («t-i)J - |K - Mt-i|| 2 + A 
< 2u' t ■ (a-R*(u t -ij) +A < -eo, 



which is the desired condition dl4> . 

If ||«t — ttt_i||/||itt|| > a, define 0J2 as in d!5l >. then 
for all w[t] such that V{w[t}) > (1 + l/a 2 )w 2 = 
1 1 u t — itt-i|| 2 > ^2 and ( TBI holds. 

Combining the above two cases and letting 
O = max{f2i, ^2}, we see that the region specified 
in Jl 3I > satisfies Lemma \l\ and therefore the queueing 
system is stable under any average arrival rate vector 
a S int C. □ 
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